1. Field of the Invention
The present invention relates to a video signal processing apparatus and, more particularly, to a video signal processing apparatus improving image quality when illuminance is low.
2. Description of the Background Art
When an image is picked up by a video terminal having a camera device using an image pickup device such as a CCD (Charge Coupled Device) under low illuminance condition, for example indoors or at night outside, video images picked up by the camera device may be extremely dark or suffer from much noise.
The picked up video images become extremely dark as sensitivity of the image pickup device such as the CCD employed in the camera device is low. As semiconductors have been developed to be smaller and smaller recently, the CCD has also been reduced in size, and hence the number of signal electrons handled by one pixel has been reduced. Therefore, the output level of the CCD for an image picked up under dark condition becomes small, resulting in low sensitivity.
The cause of extremely conspicuous noise is that shot noise generated at the time of photoelectric conversion becomes innegligible, as the number of signal electrons per pixel has been reduced.
The following methods have been conventionally known as effective in improving sensitivity.
a) To enlarge aperture ratio of iris.
b) To set shutter speed slower and to make exposure time longer.
c) To improve signal level by signal processing such as AGC (Auto Gain Control).
d) To improve signal level by signal processing through frame addition method.
As to the method a), that is, to increase aperture ratio of an iris, the larger the aperture ratio, the larger lens must be mounted. When an image is picked up under bright condition, a camera sensor would be saturated when the aperture ratio is large. Therefore, there is a limit in increasing the aperture ratio.
When automatic iris mechanism is mounted on the lens, the lens body will be increased in size, and the cost is also increased. Therefore, for a video terminal of which cost or power consumption is of priority, a fixed iris having relatively small aperture ratio must be used.
As to the method b), most of the image pickup devices such as the CCD allows free adjustment of shutter speed, by variable electronic shutter control. When an NTSC (National Television System Committee) video signal is to be generated, there is a limit of shutter speed in accordance with NTSC standard, which requires video signals of thirty (30) frames per second. Therefore, most image pickup devices allow control of shutter speed to the minimum (longest exposure time) of {fraction (1/30)} sec.
A specific example of the prior art utilizing the method c) is shown in FIG. 15. Referring to FIG. 15, an AGC circuit includes a GCA (Gain Control Amplifier) circuit 1 connected to an output of D/A (Digital to Analog) converter 4, which will be described later, an A/D (Analog to Digital) converter 2 connected to an output of GCA circuit 1, a signal level detecting circuit 47 connected to an output of A/D converter 2, and a D/A converter 4 connected to an output of signal level detecting circuit 47.
A video signal output from an image pickup device is first input to GCA circuit 1, as an input signal to AGC circuit. GCA circuit 1 amplifies the input video signal, in accordance with a gain control signal G1xe2x80x2, which will be described later, obtained from D/A converter 4.
The amplified video signal is input to A/D converter 2 and converted to a digital video signal. The video signal which has been converted to the digital video signal is output as an output video signal of AGC circuit.
The output video signal is at the same time supplied to signal level detecting circuit 47. Signal level detecting circuit 47 detects the level of the present video signal, from the output video signal.
The result of level detection is input to D/A converter 4 as a gain control coefficient K1xe2x80x2. D/A converter 4 converts gain control coefficient K1xe2x80x2 to an analog control signal, and provides the gain control signal G1xe2x80x2.
The output gain control signal G1xe2x80x2 is fed back to GCA circuit 1, and automatic gain control takes place.
Referring to FIG. 16, the change in gain control coefficient K1xe2x80x2 output from signal level detecting circuit 47 when illuminance gradually changes from high (blight) to low (dark) will be described.
When the illuminance is high, the signal level is high, and therefore a small gain control coefficient K1xe2x80x2=min is output to provide low gain.
As the illuminance gradually lowers, the signal level becomes higher, and therefore gain control coefficient gradually increases to K1xe2x80x2=max to attain higher gain.
When the luminance attains still lower (darker) after gain control coefficient K1xe2x80x2 attains to K1xe2x80x2=max, the maximum value of gain control coefficient K1xe2x80x2=max is maintained. Therefore, the signal level becomes lower.
As to the method d), that is, frame addition method, Japanese Patent Laying-Open No. 5-344417 entitled xe2x80x9cHigh Sensitivity Cameraxe2x80x9d proposes a video signal processing apparatus which improves output level of the image pickup device under low illuminance.
Referring to FIG. 17, a frame addition circuit constituting a part of a conventional video signal processing apparatus includes an A/D converter 2, an image memory 5 connected to an output of A/D converter 2, and an adder 6 connected to image memory 5 and to an output of A/D converter 2.
The video signal output from the image pickup device is first input to A/D converter 2. A/D converter 2 converts the input video signal to a digital video signal, and outputs the digital video signal to image memory 5 and adder 6. Image memory 5 has a storage capacity of one frame, and operates as a one frame delay circuit. The video signal delayed by one frame and the present video signal not delayed, output from A/D converter 2 are input to adder 6 and added to each other. The added video signal is output as an output video signal of the frame addition circuit.
The AGC circuit implementing the method c) shown in FIG. 15 is capable of amplifying a video signal of a very low level picked up with low illuminance always to a constant level, by automatic gain control in accordance with the present video signal level. Therefore, the AGC circuit has been used in many video terminals.
When a video signal of a very low level picked up with low illuminance is amplified with high gain, however, shot noise is also amplified, which means that the noise component is increased, degrading S/N (signal/noise) ratio.
Further, the frame addition circuit implementing the method d) shown in FIG. 17 provides video signals of twice the amplitude level, as a video signal delayed by one frame and a present video signal not delayed are added.
Further, the noise component is a shot noise with low frame correlation. Therefore, when frames with noise components are added, in most cases the addition is between a pixel with noise component and another pixel without noise component. Therefore, the original video signal is doubled by addition, while noise component is not added but averaged in the added two frames. Therefore, noise component level can be suppressed to some extent.
As still images are added and averaged, the signal to noise ratio S/N of each image can be improved in the following manner.
Generally, the S/N ratio of an optical system is represented by the following equation.       S    N    =            signal      ⁢                        xe2x80x83                ⁢                  xe2x80x83                    (              P        ,        M            )                      quantum        ⁢                  xe2x80x83                ⁢        noise        ⁢                  xe2x80x83                ⁢                  (                      P            ,            M                    )                    +              thermal        ⁢                  xe2x80x83                ⁢        noise            +              system        ⁢                  xe2x80x83                ⁢        noise            
Namely, the signal and the quantum noise are increased in accordance with the input photoelectric power P and the sensor magnification M. On the other hand, thermal noise is not dependent thereon.
Accordingly, thermal noise is negligible when an image is picked up using a sensor having high magnification with sufficient optical input.
When the optical input is weak (dark), however, these noises cannot be entirely neglected.
The reason for this is as follows. The quantum noise (shot noise) has statistical fluctuation, as the incident light has the nature of particles.
The fluctuation has Poisson distribution, and when an average value of the number of photons incident on one pixel is represented as n, standard deviation is given by {square root over (n)}, which represents fluctuation, or noise. Accordingly, the following definition is given.       S    N    =                    average        ⁢                  xe2x80x83                ⁢        value                    standard        ⁢                  xe2x80x83                ⁢        deviation              =                  n                  n                    =              n            
Accordingly, when the number of photons is increased, that is, when integration (addition) is performed, n increases, and hence S/N ratio increases accordingly.
Addition and averaging of thermal noise is considered addition of sources of mutually not correlated noises in normal distribution (correlation coefficient y=0), and noise power is added directly.
The system noise refers to the noise inherent to the system such as quantum noise introduced by A/D converter, and the system noise has different nature dependent on the cause of generation. Therefore, general discussion is impossible.
As to the quantum noise mentioned above, by AGC, signal intensity can be made uniform, and therefore quantum noise derived from A/D converter, for example, can be suppressed.
By addition (averaging) described above, noise corresponding to quantum noise can be reduced.
The noise component itself, however, still exists, and therefore when the degree of image quality degradation in terms of S/N ratio is monitored for a prescribed time period, for example over several seconds, improvement in S/N ratio cannot be attained as in the case of AGC circuit of c).
In other words, the effect of improvement is not observed when continuous images are viewed as a whole.
When the level improvement of twice or more is to be attained by the frame addition circuit, an image memory 7 and an adder 8 are added, as shown in FIG. 18. In this manner, a video signal delayed by two frames, a video signal delayed by one frame and the present video signal not delayed are added, so that a video signal having three times the amplitude level is obtained.
By adding a video signal preceding one frame further, it is possible to obtain a video signal of which magnification is increased to 2, 3, 4 . . . . What can be attained, however, is only the level improvement of integer multiple, and multistep control performed on tones of which accuracy is determined by the number of bits of D/A converter as in AGC circuit of c) is impossible. Therefore, what is attained is always level improvement to a prescribed extent only.
The present invention was made to solve the above described problems and its object is to provide a video signal processing apparatus capable of improving signal level while reducing noise component by a process of adding and averaging frames, no matter a video signal of what illuminance is input.
Another object of the present invention is to provide a video signal processing apparatus capable of improving signal level while reducing noise component by a process of adding and averaging lines and frames, no matter a video signal of what illuminance is input.
A still further object of the present invention is to provide a video signal processing apparatus capable of improving signal level while reducing noise component, no matter a scaled video signal of what illuminance is input.
A still further object of the present invention is to provide a video signal processing apparatus capable of optimal control of filter parameter for noise reduction, no matter a video signal of what illuminance is input.
According to an aspect of the present invention, a video signal processing apparatus includes: a video signal amplifying circuit amplifying an input video signal and outputting a video signal of a predetermined image size in accordance with a gain control coefficient; a frame addition circuit connected to the video signal amplifying circuit for adding by a predetermined number of frames, the outputs of the video signal amplifying circuit; a first signal level detecting circuit connected to the video signal amplifying circuit, for calculating the gain control coefficient and a multiplication coefficient in accordance with an output of the video signal amplifying circuit; and a first multiplier connected to the frame addition circuit and the first signal level detecting circuit, receiving as inputs the output of the frame addition circuit and the multiplication coefficient.
The gain control coefficient and the multiplication coefficient are calculated in accordance with the brightness of the video signal. Therefore, when a video signal which is within a relatively bright illuminance range is input, adding and averaging process is performed by the frame addition circuit and the multiplier. Thus noise component is reduced. When a video signal which is within a relatively dark illuminance range is input, the multiplication coefficient is increased as the signal level of the video signal decreases. Thus the process of adding and averaging frames is performed, so that the signal level can be improved while the noise component is reduced. When a video signal which is within darker illuminance range is input, the multiplication coefficient is made closer to 1 to perform the process of adding faces. Thus the signal level is improved.
Preferably, the frame addition circuit includes: a line memory connected to the video signal amplifying circuit for video signals of one line; a video memory connected to the video signal amplifying circuit for video signals of one frame; a first adder connected to the video signal amplifying circuit and the line memory and receiving as inputs an output of the video signal amplifying circuit and the video signals stored in the line memory; and a second adder connected to the first adder and the image memory and receiving as inputs an output of the first adder and the video signals stored in the image memory.
The gain control coefficient and the multiplication coefficient are calculated in accordance with the brightness of the video signal. Therefore, when a video signal which is within a relatively blight illuminance range is input, adding and averaging processes between lines and between frames are performed by the first and second addition circuits and the multiplier. Thus, noise component is reduced. When a video signal which is within a relatively dark illuminance range is input, the multiplication coefficient is increased as the signal level of the video signal decreases. Thus adding and averaging processes between lines and between frames are performed, so that the signal level can be improved while the noise component is reduced. When a video signal which belongs to darker illuminance range is input, the multiplication coefficient is made closer to 1, and adding processes between lines and between frames are performed. Thus, signal level is improved.
More preferably, the video signal amplifying circuit includes: an amplifying circuit for amplifying the input video signal; and a first scaling circuit connected to the amplifying circuit for scaling an image data constituted by the outputs of the amplifying circuit at a prescribed magnification. The frame addition circuit includes: a first image memory connected to the first scaling circuit and storing a predetermined number of outputs of the first scaling circuit; and an accumulation adder for accumulating and adding the output of the first scaling circuit and the prescribed number of outputs of the first scaling circuit stored in the first image memory.
The gain control coefficient and the multiplication coefficient are calculated in accordance with the brightness of the video signal. The video signal is scaled by a predetermined magnification, by the first scaling circuit. Therefore, when a video signal within a relatively bright illuminance range is input, adding and averaging process among a predetermined number of frames is performed by the accumulation adder. Thus, noise component is reduced. When a video signal within a relatively dark illuminance range is input, the multiplication coefficient is made larger as the signal level of the video signal decreases. Thus, adding and averaging process between frames is performed, so that the signal level can be improved while the noise component is reduced. When a video signal of a darker illuminance range is input, the multiplication coefficient is made closer to 1, and accumulation and adding process between frames is performed. Thus the signal level is improved.
More preferably, the first signal level detecting circuit includes a second signal level detecting circuit connected to the video signal amplifying circuit and calculating the gain control coefficient, the multiplication coefficient and a first filter parameter coefficient in accordance with an output of the video signal amplifying circuit. The video signal processing circuit further includes a noise reduction circuit connected to the first multiplier and to the second signal level detecting circuit and performing noise reduction of an output of the first multiplier in accordance with the first filter parameter coefficient and the output of the first multiplier.
The first filter parameter coefficient is calculated in accordance with the brightness of the video signal. More specifically, when a video signal within a relatively bright illuminance range is input, the first filter parameter coefficient is calculated such that by the amount of reduction of the noise component realized by the adding and averaging process, filter parameter of the noise reduction circuit is weakened. In a relatively dark illuminance range, the first filter parameter coefficient is calculated such that the filter parameter of the noise reduction circuit is gradually increased as the effect of noise component reduction attained by the adding and averaging process decreases. In a darker illuminance range, the first filter parameter coefficient is calculated to maximize the filter parameter of the noise reduction circuit. In this manner, the filter parameter of the noise reduction circuit is automatically controlled, realizing optimal noise reduction of the video signal.
The foregoing and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.